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METHOD AND SYSTEM FOR DEVICE CONTROL 

FIELD OF INVENTION 

The present invention relates generally to device control and, more particularly, 

to a method and system for enabling electrical or electronic devices to be controlled 
5 through distributed computing systems. It has specific application to the operation of a 
device involving a printed form-based user interface. 

CO-PENDING APPLICATIONS 

Various methods, systems and apparatus relating to the present invention are 

disclosed in the follovring co-pending applications filed by the applicant or assignee of 

10 the present invention simultaneously with the present invention: 

NPAOllUS, NPA031US, NPA040US, NPA046US, NPA053US, NPA059US, 
NPA064US, NPB006US, NPS004US, NPS008US, NPS013US, NPS024US, 
NPPCl, UPOIUS, UP02US, UP03US, UP04US, UP05US 

15 The disclosures of these co-pending applications are incorporated herein by cross- 
reference. Each appUcation is temporarily identified by its docket number. This will be 
replaced by the corresponding USSN when available. 

Various methods, systems and apparatus relating to the present invention are 
disclosed in the following co-pending applications filed by the applicant or assignee of 

20 the present invention on 15 September 2000: 

NPA024US, NPA025US, NPA047US, NPA049US 

The disclosures of these co-pending applications are incorporated herein by cross- 
reference. Each application is temporarily identified by its docket number. This will be 
25 replaced by the corresponding USSN when available. 

Various methods, systems and apparatus relating to the present invention are 
disclosed in the following co-pending applications filed by the applicant or assignee of 
the present invention on 30 June 2000: 

NPA014US, NPA015US, NPA022US, NPA026US, NPA038US, NPA041US, 
30 NPA050US, NPA051US, NPA052US, NPA063US, NPA065US, NPA067US, 
NPA068US, NPA069US, NPA071US, NPA072US, NPB003US, NPB004US, 
NPB005US, NPP019US, PEC04US, PEC05US, PEC06US, PEC07US 
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10 



15 



20 



The disclosures of these co-pending applications are incorporated herein by cross- 
reference. Each application is temporarily identified by its docket number. This will be 
replaced by the corresponding USSN when available. 

Various methods, systems and apparatus relating to the present invention are 
disclosed in the following co-pending applications filed by the applicant or assignee of 
the present invention on 23 May 2000: 



NPAOOIUS, 

NPA008US, 

NPA018US, 

NPA048US, 

NPK004US, 

NPNOOIUS, 

NPP008US, 

NPS020US, 

NPX003US, 

IJM52US, 

MJ15US, 

PAK04US, 

PEC02US, 



NPA002US, 

NPA009US, 

NPA019US, 

NPA075US, 

NPK005US, 

NPPOOIUS, 

NPP016US, 

NPTOOIUS, 

NPX008US, 

MJIOUS, 

MJ34US, 

PAK05US, 

PEC03US 



NPA004US, 
NPAOIOUS, 
NPA020US, 
NPBOOIUS, 



NPA005US, 
NPA012US, 
NPA021US, 
NPB002US, 



NPA006US, 
NPA016US, 
NPA030US, 
NPK002US, 



NPA007US, 
NPA017US, 
NPA035US, 
NPK003US, 



NPMOOIUS, NPM002US, NPM003US, NPM004US, 
NPP003US, NPP005US, NPP006US, NPP007US, 



NPP017US, 

NPT002US, 

NPXOllUS, 

MJllUS, 

MJ47US, 

PAK06US, 



NPP018US, 

NPT003US, 

NPX014US, 

MJ12US, 

MJ58US, 

PAK07US, 



NPSOOIUS, 

NPT004US, 

NPX016US, 

MJ13US, 

MJ62US, 

PAK08US, 



NPS003US, 

NPXOOIUS, 

IJ52US, 

MJ14US, 

MJ63US, 

PECOIUS, 



The disclosures of these co-pending applications are incorporated herein by cross- 
reference. Each application is temporarily identified by its docket number. This will be 
replaced by the corresponding USSN when available. 
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BACKGROUND 

There are many electrical and electronic devices in the home and office which 
have some kind of user interface for controlling their operation. These include office 
equipment, audio and video equipment, kitchen appliances, heating and cooling systems, 
5 etc. Even light sockets have on/off switches and sometimes dimmers, and in many 
countries power outlets have on/off switches. 

Office equipment and entertainment equipment is increasingly networkable - 
i.e. devices come with interfaces to standard local-area networks, allowing their 
operation to be controlled over a network. The advent of standard wireless networking 
10 technologies such as Bluetooth may see an increase in networking support in such 
devices and appliances. 

Some networking futurists predict that even light sockets will one day be 
networked, allowing their status to be queried and their function to be controlled 
remotely. 

15 Even today, an increasing number of devices can be controlled via infrared 

remote control interfaces. Such interfaces are particularly prevalent on audio and video 
equipment and heating and cooling systems. 

SUMMARY OF INVENTION 

It is an objective of the present invention to provide a new method and system 

20 for enabling devices to be controlled through a computer system. 

In a first aspect, the present invention provides for a method of enabling a 
device to be controlled in which the device is operative to perform at least one fimction 
in response to control instructions from a computer system, the method including the 
steps of 

25 providing a control interface containing information relating to the device 

function, the control interface including coded data indicative of an identity of the 
control interface and of at least one reference point of the control interface; 

receiving, in the computer system, indicating data from a sensing device 
regarding the identity of the control interface and a position of the sensing device relative 
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to the control interface, the sensing device, when placed in an operative position relative 
to the control interface, sensing the indicating data using at least some of the coded data; 
and 

effecting, in the computer system and from the indicating data, an operation 
5 relating to at least one parameter of the control instructions. 

In a second aspect, the present invention provides for a method of enabling a 
device to be controlled in which the device is operative to perform at least one function 
in response to control instructions from a computer system, the method including the 
steps of 

10 providing a control interface containing information relating to the device 

fimction, the control interface including coded data indicative of at least one parameter 
of the control instructions; 

receiving, in the computer system, data from a sensing device relating to said at 
least one parameter and regarding movement of the sensing device relative to the control 
1 5 interface, the sensing device, when moved relative to the control interface, sensing the 
data regarding said at least one parameter using at least some of the coded data and 
generating the data regarding its own movement relative to the control interface; and 

interpreting, in the computer system, said movement of the sensing device as it 
relates to said at least one parameter. 

20 hi a third aspect, the present invention provides for a method of enabling a 

device to be controlled in which the device is operative to perform at least one fimction 
in response to control instructions from a computer system, the method including the 
steps of 

providing a user with a control interface including coded data indicative of an 
25 identity of the control interface; 

receiving, in the computer system, data from a sensing device regarding an 
identity of the user and regarding the identity of the control interface, the sensing device 
containing the data regarding the identity of the user and sensing the data regardmg the 
identity of the control interface using at least some of the coded data; and 

30 effecting, in the computer system and from the data regarding the identity of 
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the user and the identity of the control instruction, an operation relating to at least one 
parameter of the control instructions. 

In a fourth aspect, the present invention provides for a system for enabling at 
least one function of a device to be controlled, the system including 

5 a control interface containing information relating to the device function, the 

control interface including coded data indicative of an identity of the control interface 
and of at least one reference point of the control interface; and 

a computer system incorporating control instructions operative to cause said 
device to perform said function and operative to receive indicating data from a sensing 
10 device for effecting an operation relating to at least one parameter of the control 
instructions, the indicating data being indicative of the identity of the control interface 
and a position of the sensing device relative to the control interface, the sensing device 
sensing the indicating data using at least some of the coded data. 

In a fifth aspect, the present invention provides for a system for enabling at 
1 5 least one function of a device to be controlled in response to control instructions, the 
system including 

a control interface containing information relating to the device function, the 
control interface including coded data indicative of at least one parameter of the control 
instructions; and 

20 a computer system incorporating said control instructions, and operative to 

receive data from a sensing device regarding said at least one parameter and regarding 
movement of the sensing device relative to the control interface, and for interpreting said 
movement of the sensing device as it relates to said at least one parameter, the sensing 
device, when moved relative to the control interface, sensing the data regarding said at 

25 least one parameter using at least some of the coded data and generating the data 
regarding its own movement relative to the control interface. 

In a sixth aspect, the present invention provides for a system for enabling at 
least one function of a device to be controlled, the system including 

a control interface including coded data indicative of an identity of the control 

30 interface; and 
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a computer system incorporating control instructions to cause said device to 
perform said function, and operative to receive from a sensing device data regarding an 
identity of the user and the identity of the control interface, and for effecting, from said 
received data, an operation relating to a parameter of said control instructions, the 
5 sensing device containing the data regarding the identity of the user and sensing the data 
regarding the identity of the control interface using at least some of the coded data. 

Accordingly, the present invention provides a system and a method which 
utilizes one or more device control interfaces capable of interacting with a computer 
system to control at least one fimction of at least one device. Whilst the novel method 
10 and system of the present invention may be used in conjunction with a single computer 
system, in a particularly preferred form it is designed to operate over a computer 
network, such as the Internet. 

Physically, the device control interface is disposed on a surface medium which 
may be of any suitable structure. However, in a preferred arrangement, the form 

1 5 comprises sheet material such as paper or the like which has the coded data printed on it 
and which allows it to interact with the computer system. The coded data is detectable 
preferably, but not exclusively, outside the visible spectrum, thereby enabling it to be 
machine-readable but substantially invisible to the human eye. The device control may 
also include visible material which provides information to a user, such as the 

20 application or purpose of the device control, and which visible information may be 
registered or correlate in position v^th the relevant hidden coded data. 

The system also includes a sensing device to convey data from the device 
control to the computer system, and in some instances, to contribute additional data 
Again, the sensing device may take a variety of forms but is preferably compact and 

25 easily portable. In one arrangement, the sensing device is configured as a pen which is 
designed to be able to physically mark the interactive form as well as to selectively 
enable the coded data from the form to be read and transmitted to the computer system. 
The coded data then provides control information, configured such that designation 
thereof by a user causes instructions to be applied to the software running on the 

30 computer system or network. Alternatively, the sensing device may be embedded in a 
ring, hand glove or finger glove worn by a user. It could also be embedded in custom 
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eyewear, with interaction controlled by gaze. 

The nature of the interaction between the device control interface and the 
sensing device and the data that each contributes to the computer system may vary. In 
one arrangement, the coded data on the form is indicative of the identity of the form and 
5 of at least one reference point on that form. In another embodiment, the interactive form 
includes coded data which is indicative of a parameter of the form, whereas the sensing 
device is operative to provide data regarding its own movement relative to that form to 
the computer system together with coded data from the form. In yet another arrangement, 
the form includes the coded data which at least identifies the form, and the sensing 
10 device is designed to provide, to the computer system, data based on the form coded 
data, and also on data which identifies the user of the device. 

The system and method also preferably employs specially designed printers to 
print the control devices. Further these printers constitute or form part of the computer 
system and are designed to receive data from the sensing device. As indicated above, the 
1 5 system and method of the invention is ideally suited to operate over a network. In this 
arrangement, the printers are fully integrated into the network and allow for printing of 
the interactive forms on demand and also for distributing of the forms using a mixture of 
multi-cast and point-cast communication protocols. 

The present invention therefore provides a novel system and method for 
20 enabling electrical or electronic devices to be controlled through a computer system 
making use of novel interactive surface media. The invention is applicable where a 
particular device is able to be remotely controlled, either via a network interface or an 
infrared interface. In these circumstaces it is possible to completely separate the user 
interface to the device firom the device itself and the user interface can then be 
25 implemented in the forms mentioned above and sensed by a sensing device carried by 
the user. 

The device control interface preferably identifies the device, and is printed with 
the user interface of the device in the form of buttons etc. Being passive, the surface 
gives no feedback to the user as to the status of the device or its user interface. Feedback 
30 can, however, be provided via the sensing device if necessary. The computer system 
resolves operations on the control interface with reference to a user interface structure 
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previously selected by the user from the device itself or its manufacturer, or designed by 
the user using elements provided by the device or its manufacturer. 

User interfaces implemented in this way have several advantages: 

once the infrastructure is in place, the cost of manufacturing the user 
5 interface is removed from the cost of manufacturing each particular device; as an 
example, even the cost of the traditional user interface to a light socket (i.e. both the 
wiring and the switch) is non-trivial; 

a user interface can be replicated by simply printing another copy 

when a user interface becomes damaged it can be easily replaced 

10 • when the firmware in a device is updated, a new user interface can be 

easily provided 

a user interface can be customized by arranging elements and printing the 

result 

• a user interface with a particular style can be chosen from an available 
1 5 set; for example, some users may prefer larger icons or a simplified function set 

• multiple user interfaces can be grouped together 

• a user interface can be provided on the device itself by the same means; 
i.e. pre-printed and attached to the device, or printed directly onto the device surface 

• access to individual user interfaces and functions can be controlled, since 
20 each sensing device is tied to a particular user's identity. 

As one example, a device control interface according to a preferred aspect of 
the invention can be printed onto or attached to the surface of a toy to provide the user 
interface to the toy. An appropriate sensing device cm then be used to interact with the 
user interface, communicating by wire or wirelessly to either a personal computer, game 
25 console, or a processor embedded in the toy itself, to produce graphical output, sound 
effects and other behavior. For example, a toy sit-in submarine may have a large user 
interface containing dozens of submarine controls. The toy comes v^th a CD-ROM 
containing an associated interactive game which runs on a PC or game console. The 
game is driven by the user interface contained in the submarine, perhaps mediated by a 
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sensing device in the form of a glove which communicates wirelessly by Bluetooth with 
standard Bluetooth hardware in the PC or game console. The PC screen or console- 
connected television monitor provides the animated part of the user interface. The device 
control interface provides the extended user interface. Toys can be customized by 
5 selectmg different user interfaces and custom user interfaces can be created by 
assembling pre-defined user interface components. 
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BRIEF DESCRIPTION OF DRAWINGS 

Preferred and other embodiments of the invention will now be described, by 

way of non-limiting example only, with reference to the accompanying drawings, in 
which: 

5 Figure 1 is a schematic of a the relationship between a sample printed netpage and its 
online page description; 

Figure 2 is a schematic view of a interaction between a netpage pen, a netpage printer, a 
netpage page server, and a netpage application server; 

Figure 3 is a schematic view of a high-level structure of a printed netpage and its online 
1 0 page description; 

Figure 4a is a plan view showing a structure of a netpage tag; 

Figure 4b is a plan view showing a relationship between a set of the tags shown in Figure 
4a and a field of view of a netpage sensing device in the form of a netpage pen; 

Figure 5a is a plan view showing an alternative structure of a netpage tag; 

1 5 Figure 5b is a plan view shoving a relationship between a set of the tags shown in Figure 
5a and a field of view of a netpage sensing device in the form of a netpage pen; 

Figure 5c is a plan view showing an arrangement of nine of the tags shown in Figure 5a 
where targets are shared between adjacent tags; 

Figure 5d is a plan view showing the interleaving and rotation of the symbols of the four 
20 codewords of the tag shown in Figure 5a; 

Figure 6 is a schematic view of a set of user mterface flow document icons; 
Figure 7 is a schematic view of a set of user interface page layout element icons; 
Figure 8 is a schematic view of device control interactions; 
Figure 9 is a schematic view of a device control class diagram; 
25 Figure 10 is a schematic view of a default device class diagram; 

Figure 1 1 is a schematic view of a device information class diagram; 
Figure 12 is a schematic view of a device channel class diagram; 
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Figiire 13 is a schematic view of a remote control class diagram; 

Figure 15 is a schematic view of a program schedule class diagram; 

Figure 16 is a schematic view of a device setup user interface flow; 

Figure 17 is a schematic view of a setup details user interface flow; 
5 Figure 1 8 is a schematic view of a device list user interface flow; 

Figure 19 is a schematic view of an add device user interface flow; 

Figure 20 is a schematic view of a record program user interface flow; 

Figure 21 is a detailed view of a device setup main page; 

Figure 22 is a schematic view of a device setup details; 
1 0 Figure 23 is a schematic view of a device list; 

Figure 24 is a schematic view of a imiversal remote control form; 

Figure 25 is a schematic view of a device specific TV remote control; 

Figure 26 is a schematic view of a universal remote control; 

Figure 27 is a schematic view of a select manufacturer list; 
1 5 Figure 28 is a schematic view of a select device type list; 

Figure 29 is a schematic view of a select model list; 

Figure 30 is a schematic view of a set defaults form; and 

Figure 3 1 is a schematic view of a program schedule. 
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DETAILED DESCRIPTION OF PREFERRED AND OTHER EMBODIMENTS 

Note: Memjet'T^ is a trademark of Silverbrook Research Pty Ltd, Australia. 

In the preferred embodiment, the invention is configured to work with the 
netpage networked computer system, a summary of which is given below and a detailed 
5 description of which is given in our earlier applications, including in particular 

applications USSN 09/ (docket no. NPT002US), USSN 09/ 

(docket no, NPSOOIUS), USSN 09/ (docket no. NPP003US), USSN 

09/ (docket no. NPA002US) and USSN 09/ (docket no. 

IJ52US). It will be appreciated that not every implementation will necessarily embody all 
10 or even most of the specific details and extensions described in these applications in 
relation to the basic system. However, the system is described in its most complete form 
to assist in vmderstanding the context in which the preferred embodiments and aspects of 
the present invention operate. 

In brief summary, the preferred form of the netpage system employs a computer 
1 5 interface in the form of a mapped surface, that is, a physical surface which contains 
references to a map of the surface maintained in a computer system. The map references 
can be queried by an appropriate sensing device. Depending upon the specific 
implementation, the map references may be encoded visibly or invisibly, and defined in 
such a way that a local query on the mapped surface yields an unambiguous map 
20 reference both within the map and among different maps. The computer system can 
contain information about features on the mapped surface, and such information can be 
retrieved based on map references supplied by a sensing device used v^th the mapped 
surface. The information thus retrieved can take the form of actions which are initiated 
by the computer system on behalf of the operator in response to the operator's interaction 
25 with the surface features. 

In its preferred form, the netpage system relies on the production of, and human 
interaction with, netpages. These are pages of text, graphics and unages printed on 
ordinary paper or other media, but which work like interactive web pages. Information is 
encoded on each page using ink which is substantially invisible to the unaided human 
30 eye. The ink, however, and thereby the coded data, can be sensed by an optically imaging 
pen and transmitted to the netpage system. 
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In the preferred form, active buttons and hyperlinks on each page can be 
clicked with the pen to request information from the network or to signal preferences to a 
network server. In one embodiment, text written by hand on a netpage is automatically 
recognized and converted to computer text in the netpage system, allowing forms to be 
5 filled in. In other embodiments, signatures recorded on a netpage are automatically 
verified, allowing e-commerce transactions to be securely authorized. 

As illustrated in Figure 1, a printed netpage 1 can represent a interactive form 
which can be filled in by the user both physically, on the printed page, and 
"electronically", via communication between the pen and the netpage system. The 

10 example shows a "Request form containing name and address fields and a submit 
button. The netpage consists of graphic data 2 printed using visible ink, and coded data 3 
printed as a collection of tags 4 using invisible ink. The corresponding page description 
5, stored on the netpage network, describes the individual elements of the netpage. In 
particular it describes the type and spatial extent (zone) of each interactive element (i.e. 

1 5 text field or button in the example), to allow the netpage system to correctly interpret 
input via the netpage. The submit button 6, for example, has a zone 7 which corresponds 
to the spatial extent of the corresponding graphic 8. 

As illustrated in Figure 2, the netpage pen 101, a preferred form of which is 

described in our earlier application USSN 09/ (docket no. NPSOOIUS), 

20 works in conjunction with a netpage printer 601, an Internet-connected printing 
appliance for home, office or mobile use. The pen is wireless and communicates securely 
with the netpage printer via a short-range radio link 9. 

The netpage printer 601, preferred forms of which are described in our earlier 
apphcation USSN 09/ (docket no. NPP003US) and our co-filed application 

25 USSN 09/ (docket no. NPS024US), is able to deliver, periodically or on 

demand, personalized newspapers, magazines, catalogs, brochures and other 
publications, all printed at high quality as interactive netpages. Unlike a personal 
computer, the netpage printer is an appliance which can be, for example, wall-moimted 
adjacent to an area where the morning news is first consumed, such as in a user's 

30 kitchen, near a breakfast table, or near the household's point of departure for the day. It 
also comes in tabletop, desktop, portable and miniature versions. 
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Netpages printed at their point of consumption combine the ease-of-use of 
paper with the timeUness and interactivity of an interactive medium. 

As shown in Figure 2, the netpage pen 101 interacts with the coded data on a 
printed netpage 1 and communicates, via a short-range radio link 9, the interaction to a 
5 netpage printer. The printer 601 sends the interaction to the relevant netpage page server 
10 for interpretation. In appropriate circumstances, the page server sends a corresponding 
message to application computer software running on a netpage application server 13. 
The application server may in turn send a response which is printed on the originating 
printer. 

10 The netpage system is made considerably more convenient in the preferred 

embodiment by being used in conjimction with high-speed microelectromechanical 
system (MEMS) based Inkjet (Memjet™) printers, for example as described in our 

earlier application USSN 09/ (docket no. IJ52US). In the preferred form of 

this technology, relatively high-speed and high-quality printing is made more affordable 

15 to consumers. In its preferred form, a netpage publication has the physical characteristics 
of a traditional newsmagazine, such as a set of letter-size glossy pages printed in full 
color on both sides, bound together for easy navigation and comfortable handling. 

The netpage printer exploits the growing availability of broadband Internet 
access. The netpage printer can also operate with slower connections, but with longer 

20 delivery times and lower image quality. The netpage system can also be enabled using 
existing consumer Inkjet and laser printers, although the system will operate more slowly 
and will therefore be less acceptable from a consimier's point of view. In other 
embodiments, the netpage system is hosted on a private intranet. In still other 
embodiments, the netpage system is hosted on a single computer or computer-enabled 

25 device, such as a printer. 

Netpage publication servers 14 on the netpage network are configured to 
deliver print-quality publications to netpage printers. Periodical publications are 
delivered automatically to subscribing netpage printers via pointcasting and multicasting 
Intemet protocols. Personalized publications are filtered and formatted according to 
30 individual user profiles. 

A netpage printer can be configured to support any number of pens, and a pen 
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can work with any number of netpage printers. In the preferred implementation, each 
netpage pen has a unique identifier. A household may have a collection of colored 
netpage pens, one assigned to each member of the family. This allows each user to 
maintain a distinct profile with respect to a netpage publication server or application 
5 server. 

A netpage pen can also be registered with a netpage registration server 1 1 and 
linked to one or more payment card accounts. This allows e-commerce payments to be 
securely authorized using the netpage pen. The netpage registration server compares the 
signature captured by the netpage pen with a previously registered signature, allowing it 
10 to authenticate the user's identity to an e-commerce server. Other biometrics can also be 
used to verify identity. A version of the netpage pen includes fingerprint scanning, 
verified in a similar way by the netpage registration server. 

Although a netpage printer may deliver periodicals such as the morning 
newspaper without user intervention, it can be configured never to deliver imsolicited 
1 5 jimk mail. In its preferred form, it only delivers periodicals fi'om subscribed or otherwise 
authorized soxirces. In this respect, the netpage printer is unlike a fax machine or e-mail 
account which is visible to any junk mailer who knows the telephone number or e-mail 
address. 

Each object model in the system is described using a Unified Modeling 
20 Language (UML) class diagram. A class diagram consists of a set of object classes 
coimected by relationships, and two kinds of relationships are of interest here: 
associations and generalizations. An association represents some kind of relationship 
between objects, i.e. between instances of classes. A generalization relates actual classes, 
and can be understood in the following way: if a class is thought of as the set of all 
25 objects of that class, and class A is a generalization of class B, then B is simply a subset 
of A. Each class is drawn as a rectangle labelled with the name of the class. It contains a 
list of the attributes of the class, separated from the name by a horizontal line, and a list 
of the operations of the class, separated from the attribute list by a horizontal line. In the 
class diagrams which follow, however, operations are never modelled. An association is 
30 drawn as a line joining two classes, optionally labelled at either end with the multiplicity 
of the association. The default multiplicity is one. An asterisk (*) indicates a multiplicity 
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of "many", i.e. zero or more. Each association is optionally labelled with its name, and is 
also optionally labelled at either end with the role of the corresponding class. An open 
diamond indicates an aggregation association ("is-part-of ), and is drawn at the 
aggregator end of the association line. A generalization relationship ("is-a'') is drawn as a 
5 solid line joining two classes, with an arrow (in the form of an open triangle) at the 
generalization end. When a class diagram is broken up into multiple diagrams, any class 
which is duplicated is shown with a dashed outline in all but the main diagram which 
defines it. It is shown with attributes only where it is defined. 

Netpages are the foundation on which a netpage network is built. They provide 
10 a paper-based user interface to published information and interactive services. A netpage 
consists of a printed page (or other surface region) invisibly tagged with references to an 
online description of the page. The online page description is maintained persistently by 
a netpage page server. The page description describes the visible layout and content of 
the page, including text, graphics and images. It also describes the input elements on the 
1 5 page, including buttons, hyperlinks, and input fields. A netpage allows markings made 
with a netpage pen on its surface to be simultaneously captured and processed by the 
netpage system. 

Multiple netpages can share the same page description. However, to allow 
input through otherwise identical pages to be distinguished, each netpage is assigned a 
20 unique page identifier. This page ID has sufficient precision to distinguish between a 
very large number of netpages. 

Each reference to the page description is encoded in a printed tag. The tag 
identifies the unique page on which it appears, and thereby indirectly identifies the page 
description. The tag also identifies its own position on the page. Characteristics of the 
25 tags are described in more detail below. 

Tags are printed in infi-ared-absorptive ink on any substrate which is infrared- 
reflective, such as ordinary paper. Near-infrared wavelengths are invisible to the human 
eye but are easily sensed by a solid-state image sensor with an appropriate filter. 

A tag is sensed by an area image sensor in the netpage pen, and the tag data is 
30 transmitted to the netpage system via the nearest netpage printer. The pen is wireless and 
communicates with the netpage printer via a short-range radio link. Tags are sufficiently 
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small and densely arranged that the pen can reliably image at least one tag even on a 
single click on the page. It is important that the pen recognize the page ID and position 
on every interaction with the page, since the interaction is stateless. Tags are error- 
correctably encoded to make them partially tolerant to surface damage. 

5 The netpage page server maintains a unique page instance for each printed 

netpage, allowing it to maintain a distinct set of user-supplied values for input fields in 
the page description for each printed netpage. 

The relationship between the page description, the page instance, and the 
printed netpage is shown in Figure 3. The printed netpage may be part of a printed 
10 netpage document 45. The page instance is associated with both the netpage printer 
which printed it and, if known, the netpage user who requested it. 

In a preferred form, each tag identifies the region in which it appears, and the 
location of that tag within the region. A tag may also contain flags which relate to the 
region as a whole or to the tag. One or more flag bits may, for example, signal a tag 
15 sensing device to provide feedback indicative of a fimction associated v^th the 
immediate area of the tag, without the sensing device having to refer to a description of 
the region. A netpage pen may, for example, illuminate an "active area" LED when in 
the zone of a hyperlink. 

In a preferred embodiment, each tag contains an easily recognized invariant 
20 structure which aids initial detection, and which assists in minimizing the effect of any 
warp induced by the surface or by the sensing process. The tags preferably tile the entire 
page, and are sufficiently small and densely arranged that the pen can reliably image at 
least one tag even on a single click on the page. It is important that the pen recognize the 
page ID and position on every interaction with the page, since the interaction is stateless. 

25 In a preferred embodiment, the region to which a tag refers coincides with an 

entire page, and the region ID encoded in the tag is therefore synonymous with the page 
ID of the page on which the tag appears. In other embodiments, the region to which a tag 
refers can be an arbitrary subregion of a page or other surface. For example, it can 
coincide with the zone of an interactive element, in which case the region ID can directly 

30 identify the interactive element. 
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Each tag contains typically contains 16 bits of tag ID, at least 90 bits of region 
ID, and a number of flag bits. Assuming a maximimi tag density of 64 per square inch, a 
16-bit tag ID supports a region size of up to 1024 square inches. Larger regions can be 
mapped continuously without increasing the tag ID precision simply by using abutting 
5 regions and maps. The distinction between a region ID and a tag ID is mostly one of 
convenience. For most purposes the concatenation of the two can be considered as a 
globally unique tag ID. Conversely, it may also be convenient to introduce structure into 
the tag ID, for example to dejSne the x and y coordinates of the tag, A 90-bit region ID 
allows 2^^ (-10^^ or a thousand trillion trillion) different regions to be uniquely 
1 0 identified. Tags may also contain type information, and a region may be tagged with a 
mixture of tag types. For example, a region may be tagged with one set of tags encoding 
X coordinates and another set, interleaved with the first, encoding y coordinates. 

In one embodiment, 120 bits of tag data are redundantly encoded using a (15, 
5) Reed-Solomon code. This yields 360 encoded bits consisting of 6 codewords of 15 4- 

15 bit symbols each. The (15, 5) code allows up to 5 symbol errors to be corrected per 
codeword, i.e. it is tolerant of a symbol error rate of up to 33% per codeword. Each 4-bit 
symbol is represented in a spatially coherent way in the tag, and the symbols of the six 
codewords are interleaved spatially within the tag. This ensures that a burst error (an 
error affecting multiple spatially adjacent bits) damages a minimum number of symbols 

20 overall and a minimum number of symbols in any one codeword, thus maximising the 
likeUhood that the burst error can be fully corrected. 

Any suitable error-correcting code code can be used in place of a (15, 5) Reed- 
Solomon code, for example a Reed-Solomon code with more or less redundancy, with 
the same or different symbol and codeword sizes; another block code; or a different kind 
25 of code, such as a convolutional code (see, for example, Stephen B. Wicker, Error 
Control Systems for Digital Communication and Storage, Prentice-Hall 1995, the 
contents of which a herein incorporated by cross-reference). 

One embodiment of the physical representation of the tag, shown in Figure 4a 

and described in our earlier application USSN 09/ (docket no. NPT002US), 

30 includes fixed target structures 15, 16, 17 and variable data areas 18. The fixed target 
structures allow a sensing device such as the netpage pen to detect the tag and infer its 
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three-dimensional orientation relative to the sensor. The data areas contain 
representations of the individual bits of the encoded tag data. To maximise its size, each 
data bit is represented by a radial wedge in the form of an area bounded by two radial 
lines and two concentric circular arcs. Each wedge has a minimum dimension of 8 dots 
5 at 1600 dpi and is designed so that its base (its inner arc), is at least equal to this 
minimum dimension. The height of the wedge in the radial direction is always equal to 
the minimum dimension. Each 4-bit data symbol is represented by an array of 2x2 
wedges. The fifteen 4-bit data symbols of each of the six codewords are allocated to the 
four concentric symbol rings 1 8a to 1 8d in interleaved fashion. Symbols are allocated 
10 alternately in circular progression aroimd the tag. The interleaving is designed to 
maximise the average spatial distance between any two symbols of the same codeword. 

hi order to support "single-click" interaction with a tagged region via a sensing 
device, the sensing device must be able to see at least one entire tag in its field of view 
no matter where in the region or at what orientation it is positioned. The required 
1 5 diameter of the field of view of the sensing device is therefore a fimction of the size and 
spacing of the tags. Assuming a circular tag shape, the minimum diameter of the sensor 
field of view 193 is obtained when the tags are tiled on a equilateral triangular grid, as 
shown in Figure 4b. 

The tag structure just described is designed to allow both regular tilings of 
20 planar surfaces and irregular tilings of non-planar surfaces. Regular tilings are not, in 
general, possible on non-planar surfaces. Li the more usual case of planar surfaces where 
regular tilings of tags are possible, i.e. surfaces such as sheets of paper and the like, more 
efficient tag structures can be used which exploit the regular nature of the tiling. 

An alternative tag structure more suited to a regular tiling is shown in Figure 
25 5a, The tag 4 is square and has four perspective targets 17. It is similar in structure to 
tags described by Bennett et al. in US Patent 5,051,746. The tag represents sixty 4-bit 
Reed-Solomon symbols 47, for a total of 240 bits. The tag represents each one bit as a 
dot 48, and each zero bit by the absence of the corresponding dot. The perspective targets 
are designed to be shared between adjacent tags, as shown in Figures 5b and 5c. Figure 
30 5b shows a square tiling of 16 tags and the corresponding minimum field of view 193, 
which must span the diagonals of two tags. Figure 5c shows a square tiling of nine tags. 
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containing all one bits for illustration pinposes. 

Using a (15, 7) Reed-Solomon code, 112 bits of tag data are redundantly 
encoded to produce 240 encoded bits. The four codewords are interleaved spatially 
within the tag to maximize resilience to burst errors. Assuming a 16-bit tag ID as before, 
5 this allows a region ID of up to 92 bits. The data-bearing dots 48 of the tag are designed 
to not overlap their neighbors, so that groups of tags cannot produce structures which 
resemble targets. This also saves ink. The perspective targets therefore allow detection of 
the tag, so further targets are not required. 

Although the tag may contain an orientation feature to allow disambiguation of 
1 0 the four possible orientations of the tag relative to the sensor, it is also possible to embed 
orientation data in the tag data. For example, the four codewords can be arranged so that 
each tag orientation contains one codeword placed at that orientation, as shown in Figure 
5d, where each symbol is labelled with the number of its codeword (1-4) and the position 
of the symbol within the codeword (A-O). Tag decoding then consists of decoding one 
1 5 codeword at each orientation. Each codeword can either contain a single bit indicating 
whether it is the first codeword, or two bits indicating which codeword it is. The latter 
approach has the advantage that if, say, the data content of only one codeword is 
required, then at most two codewords need to be decoded to obtain the desired data. This 
may be the case if the region ID is not expected to change within a stroke and is thus 
20 only decoded at the start of a stroke. Within a stroke only the codeword containing the 
tag ID is then desired. Furthermore, since the rotation of the sensing device changes 
slowly and predictably within a stroke, only one codeword typically needs to be decoded 
per frame. 

It is possible to dispense with perspective targets altogether and instead rely on 
25 the data representation being self-registering. In this case each bit value (or multi-bit 
value) is typically represented by an explicit glyph, i.e. no bit value is represented by the 
absence of a glyph. This ensures that the data grid is well-populated, and thus allows the 
grid to be reliably identified and its perspective distortion detected and subsequently 
corrected during data sampling. To allow tag boundaries to be detected, each tag data 
30 must contain a marker pattern, and these must be redundantly encoded to allow reliable 
detection. The overhead of such marker patterns is similar to the overhead of explicit 



NPA059US 



-21 - 

perspective targets. One such scheme uses dots positioned a various points relative to 
grid vertices to represent different glyphs and hence different multi-bit values (see Anoto 
Technology Description, Anoto April 2000). 

Decoding a tag results in a region ID, a tag ID, and a tag-relative pen transform. 
5 Before the tag ID and the tag-relative pen location can be translated into an absolute 
location within the tagged region, the location of the tag within the region must be 
known. This is given by a tag map, a function which maps each tag ID in a tagged region 
to a corresponding location. A tag map reflects the scheme used to tile the surface region 
with tags, and this can vary according to surface type. When multiple tagged regions 

1 0 share the same tiling scheme and the same tag numbering scheme, they can also share 
the same tag map. The tag map for a region must be retrievable via the region ID. Thus, 
given a region ID, a tag ID and a pen transform, the tag map can be retrieved, the tag ID 
can be translated into an absolute tag location within the region, and the tag-relative pen 
location can be added to the tag location to yield an absolute pen location within the 

1 5 region. 

The tag ID may have a structure which assists translation through the tag map. 
It may, for example, encoded cartesian coordinates or polar coordinates, depending on 
the surface type on which it appears. The tag ID structure is dictated by and known to the 
tag map, and tag IDs associated v^th different tag maps may therefore have different 
20 structures. 

Two distinct surface coding schemes are of interest, both of which use the tag 
structure described earlier in this section. The preferred coding scheme uses "location- 
indicating" tags as already discussed. An alternative coding scheme uses "object- 
indicating" (or "function-indicating") tags. 

25 A location-indicating tag contains a tag ID which, when translated through the 

tag map associated with the tagged region, yields a unique tag location within the region. 
The tag-relative location of the pen is added to this tag location to yield the location of 
the pen within the region. This in turn is used to determine the location of the pen 
relative to a user interface element in the page description associated with the region. 

30 Not only is the user interface element itself identified, but a location relative to the user 
interface element is identified. Location-indicating tags therefore trivially support the 
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capture of an absolute pen path in the zone of a particular user interface element. 

An object-indicating (or function-indicating) tag contains a tag ID which 
directly identifies a user interface element in the page description associated with the 
region (or equivalently^ a function). All the tags in the zone of the user interface element 
5 identify the user interface element, making them all identical and therefore 
indistinguishable. Object-indicating tags do not, therefore, support the capture of an 
absolute pen path. They do, however, support the capture of a relative pen path. So long 
as the position sampling frequency exceeds twice the encountered tag frequency, the 
displacement from one sampled pen position to the next within a stroke can be 
10 unambiguously determined. As an alternative, the netpage pen 101 can contain a pair or 
motion-sensing accelerometers, as described in our earlier application USSN 
09/ (docket no, NPSOOIUS). 

With either tagging scheme, the tags function in cooperation with associated 
visual elements on the netpage as user interactive elements in that a user can interact 
1 5 with the printed page using an appropriate sensing device in order for tag data to be read 
by the sensing device and for an appropriate response to be generated in the netpage 
system. 

Each application user interface flow is illustrated as a collection of documents 
linked by command arrows, A command arrow indicates that the target document is 
20 printed as a result of the user pressing the corresponding command button on the source 
page. Some command arrows are labelled with multiple commands separated by slashes 
(7's), indicating that any one of the specified commands causes the target docimient to 
be printed. Although multiple commands may label the same command arrow, they 
typically have different side-effects. 

25 In application terms, it is important to distinguish between netpage docimients 

and netpage forms. Documents contain printed information, as well as command buttons 
which can be pressed by the user to request further information or some other action. 
Forms, in addition to behaving like normal documents, also contain input fields which 
can be filled in by the user. They provide the system with a data input mechanism. It is 

30 also useful to distinguish between docviments which contain generic iirformation and 
documents which contain information specific to a particular interaction between the 
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user and an application. Generic documents may be pre-printed publications such as 
magazines sold at news stands or advertising posters encountered in public places. Forms 
may also be pre-printed, including, for example, subscription forms encountered in pre- 
printed publications. They may, of course, also be generated on-the-fly by a netpage 
5 printer in response to user requests. User-specific documents and forms are normally 
generated on the fly by a netpage printer in response to user requests. Figure 6 shows a 
generic document 990, a generic form 991, a user-specific document 992, and a user- 
specific form 993. 

Netpages which participate in a user interface flow are further described by 
10 abstract page layouts. A page layout may contain various kinds of elements, each of 
which has a unique style to differentiate it firom the others. As shown in Figure 7, these 
include fixed information 994, variable information 995, input fields 996, command 
buttons 997, draggable commands 998, and text hyperlinks or hypertext links 999. 

When a user interface flow is broken up into multiple diagrams, any document 
15 which is duplicated is shown with dashed outlines in all but the main diagram which 
defines it. 

8. Device Controls using the Netpage™ System 

The netpage system, then provides the facility to control electrical or electronic 
devices. If a particular device can be remotely controlled, either via a network interface 

20 or an infi*ared interface, then it is possible to completely separate the user interface to the 
device fi-om the device itself. Furthermore, the user interface can then be implemented as 
a netpage, sensed by a netpage sensing device carried by the user. The netpage sensing 
device should ideally be small, unobtrusive and portable. The netpage pen provides one 
possible form factor. Alternatively, the netpage sensing device might be embedded in a 

25 ring, hand glove or finger glove worn by the user. It could also be embedded in custom 
eyewear, with interaction controlled by gaze. 

The netpage identifies the device, and is printed with the user interface of 
the device in the form of buttons etc. Being passive, the surface giveds no feedback to 
30 the user as to the status of the device or its user interface. Feedback can, however, be 
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provided via the sensing device if necessary. The netpage base station (e.g. the netpage 
Printer) resolves operations on the netpage with reerence to a user interface structure 
previosuly selected by the user from the device itself or its manufacturer, or designed by 
the user using elements provided by the device or its manufacturer. 

5 

The following illustrates the concepts of a netpage device control in detail 
by way of a netpage device control application for audio and video equipment. Although 
specific reference is made to a netpage pen, any netpage sensing device can be used. 
Similarly, although specific reference is made to a netpage printer, any netpage base 
10 station can be used, i.e. any device which receives input from a netpage sensing device 
and either interprets the input directly or forwards the input to one or more network 
servers for interpretation. 

DEVICE CONTROL 

The netpage system provides a mechanism for controlling devices such as 

1 5 television receivers, VCRs and video-on-demand units. 

A netpage can contain printed control buttons in the style of a remote control 
unit, or printed playback control buttons associated with individual programs or movies. 
Playback controls can include play, pause, stop, start and end. An iconic image of the 
program or movie can itself act as a play button. 

20 Consumer disk-based video recorders are now available which support 

simultaneous recording and playback. They allow buffered viewing of broadcast video 
and provide some of the advantages of video-on-demaad. A daily newspaper delivered 
via the netpage system can contain links to broadcast news video captured in this way 
some time before delivery of the newspaper, perhaps under control of the netpage 

25 printer. 

With broadcast television, a printed netpage program listing can provide a 
usefiil interface to a collection of television channels, particularly if one or more 
channels are being buffered by a disk-based video recorder. With true video-on-demaad, 
a printed netpage catalog can provide a usefiil interface to a video library. 

30 The netpage pen can be augmented to work as an infrared remote control. 
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Commands generated by pen interactions with printed video playback controls can also 
be sent directly from the netpage printer to the video device, by wired or wireless control 
interface, including infrared and radio (e.g. Bluetooth). 

In the netpage system device control services are provided by one or more 
5 device control applications running on servers distributed on the network. User 
interactions with device control netpages are routed to the appropriate device control 
application via the appropriate page server using normal netpage mechanisms. 

Various device control interaction scenarios are illustrated in Figure 8. When a 
user invokes a device function on a device control netpage 500, the input is sent from the 

1 0 user's netpage pen 101 to the user's netpage printer 802, in the usual way (A). The input 
is relayed to the application 503 in the usual way (B). On receipt of the input, the 
application identifies the designated device 600 and determines the specified device 
command. If the designated device 600 is directly accessible to an application, then the 
application may send the command code directly to the device (CI). This may be the 

15 case if the device is connected to a network accessible to the application 503, e.g. a 
wired or wireless wide-area network. Otherwise the application may send the command 
code to the originating printer 802 (C2). The printer 802 may be modified to be receptive 
to device control conmiands sent to it by a device control application. If the designated 
device 600 is directly accessible to the printer 802, then the printer may send the 

20 command code directly to the device (Dl). This may be the case if the device is 
coimected to a network accessible to the printer, e.g. a wired or wireless wide-area or 
local-area network, or if the printer and device can communicate directly by wire or 
wirelessly. Otherwise the printer 502 may send the command code to the originating pen 
101 (D2). The pen 101 may be modified to be receptive to device control commands sent 

25 to it by a printer 502. The pen may be augmented with an infrared transmitter, allowing it 
to transmit the command code to the device (El). This requires the pen's infrared 
transmitter to be in line of sight of the device's infrared receiver. As an alternative to 
sending the command code to the pen 101, the printer 502 may instead send the 
command code to a remote control "pod" 504 (D3). This is a device which is installed 

30 within line of sight of the device 600. It can communicate with the printer by wire or 
wirelessly, and incorporates an infrared transmitter, allowing it to transmit the command 
code to the device 600 (E2). Because the pod 504 may be placed in a location which 
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gives it a guaranteed view of a number of devices^ it provides a useful alternative to 
augmenting the pen 101, 

Device Control Object Model 

The Device Control object model revolves around devices and device setups 

5 maintained for particular netpage users in relation to particular netpage printers. The 
Device Control class diagram is shown in Figure 9. 

Each netpage user 800 has one or more device setups 511. A user may have a 
separate device setup 511 for each netpage printer 802 they have access to. For example, 
a user 800 may have one device setup 5 1 1 at home and another at their office. Each 
1 0 device setup 511 has a name 512 assigned by the user. Each device setup 511 may have 
one or more users, but each user 800 only has access to one device setup 511 per printer 
802. 

A device setup 511 has a number of user devices 513 associated with it, one for 
each device 600 the user can control. Each user device 513 has an alias name 514 
15 assigned by the user, and has associated with it the manufacturer's device model 
information. 

Each device 513 can have a number of remote controls 515. A remote control 
can be used by any user 800 authorized for the device setup 511. A remote control 515 
has a number of remote control functions 510 associated with it, each v^th a function 
20 identifier 517. The function identifier 517 is used to look up the control code 565 for the 
function from the manufacturer 568. A remote control 515 may be device specific 518, 
imiversal 519 (i.e. control more than one device), or generic 520. The Remote Control 
class diagram is shown in Figure 13. 

A device-specific remote control 518 only operates the user device for which it 
25 was printed. 

A universal remote control (URC) 519 may operate several devices within the 
device setup 511. The user selects a device 513 on the URC 519 to change the URC 
mode and thereby to choose which device is current. The URC 519 typically only 
operates one device in each mode. 

30 A generic remote control 520 can operate any device 569 for any user. A 
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generic remote control 520 may be printed in a netpage publication such as a television 
program. A generic remote control 520 operates the default devices 521 in the device 
setup 51 1. If, for example, a household has more than one television the remote control 
520 will only operate the default television. For each device type, if no default device 
5 521 has been specified by the user then the first device of the relevant type added by the 
user is automatically made the default. A default video receiver 522 may be specified for 
a specific broadcast channel 523. This can be useful where one channel can be received 
on several of the user's devices 513. The Default Device class diagram is shown in 
Figure 10. 

1 0 When adding a device 569 to their device setup 511 the user 800 must specify 

the manufacturer 568, device type and model of the device 524. 

Each manufacturer 568 has a number of devices 569 and each device 569 has a 
number of models 524. Each device model 524 has associated with it a remote control 
interface 525, and information about the device's chaxmel, date, time and length of 

1 5 program (CDTL) programming capability 526. The remote control interface 525 contains 
information about the remote control functions 516 available for the device 569. Each 
remote control interface (RCI) fimction 516 has a function identifier 527 and the control 
code 565 to perform the function. The CDTL capability 526 specifies the information 
that can be programmed for the device. For example, a VCR can be progranmied with 

20 CDTL information, whereas a cable television receiver may only be able to be 
programmed with channel, date and time (CDT) information. The Device Information 
class diagram is shown in Figure 11. 

Each device setup 511 may have associated with it a number of program 
requests 568. A program request 568 is created when a user 800 requests a video 

25 broadcast program to be recorded. A published television program schedule 528 contains 
a number of entries 529, one for each television program to be broadcast. Associated 
with each entry is one or more CDTL information 530 records containing the channel, 
date, time and program length information for the broadcast This information can then 
be used to control the user devices 513 for unattended recording of broadcast television 

30 programs. The Program Schedule class diagram is shown in Figure 15. 

The Device Channel class diagram is illustrated in Figure 13. Each device 569 
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may have associated with it channel locality information 531, describing the broadcast 
channels available within a given locality. These may be divided into channel categories 
532, e.g. free-to-air, cable, etc. 

The user may receive a given broadcast channel 533 through several of their 
5 devices, for example firee-to-air broadcasts may be available through the television, 
VCR, and cable box. Optionally, the user has the ability to define a default video 
receiver device for each channel 533. If the user elects, via a program schedule, to record 
a program on a channel which is available through several of the user's devices, then the 
default video receiver, as specified in the user's device setup 51 1, is used. 

1 0 Device Control User Interface 

The user 800 typically obtains the netpage Device Setup main page 534 from 

the Device Control application's entry in the netpage system application catalog, 
obtained via the netpage printer's help page. The user may also bookmark the page and 
subsequently obtain it via their favorites list, or obtain it via their history list. 

1 5 The Device Control user interface flow is shown in Figure 16, 

Device Setup Main Page 

The Device Setup main page 534, shown in Figure 21, lists the device setups 

for the current netpage printer 802, and the users registered for each setup. If the 
privileges setting on the printer is not enabled then all device setups for the printer are 
20 listed. If the privileges setting is enabled then only those device setups that the user is 
authorized to see are listed. If the user is a system administrator then all device setups are 
listed. 

To print the details for a particular device setup the user presses the <Details> 
button 535 next to the desired setup. To create a new device setup, the user enters a setup 
25 name 536 and presses the <New Device Setup> button 537. Both actions print a Device 
Setup Details page 538. 

The user can press <List My Devices> 539 to print a Device List page 540 for 
the user's own device setup. The Device Setup user interface flow is shown in Figure 16. 

The Device Setup Details page 538, shovm in Figure 22, lists all the registered 
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users and devices for a device setup. If the device setup is nev^, then no registered users 
or devices are listed. 

The user can print the full details of a registered user or device by pressing the 
<Details> 541 button next to the desired entry. For a user entry a User Details page 542 
5 is printed. For a device entry a Device Details page 543 is printed showing the device 
manufacturer, device type, model, device alias, and the default devices to which this 
device is assigned. 

The user can remove a user or device from the device setup 51 1 by pressing the 
<Remove> button 544 next to the appropriate entry. The Device Setup Details page 538 
10 is reprinted with the appropriate entry removed. 

The user can add a new user to the device setup by pressing the <Add New 
User> button 545. An Add New User form 546 is printed listing all users registered on 
the current printer but not included in any device setup. A user can only be included in 
one device setup per netpage printer. 

1 5 The user can add a new device to the device setup by pressing the <Add New 

Device> button 547. A Select Manufacturer page 548 is printed. 

The Setup Details user interface flow is shown in Figure 17. 

Adding a New Device 

A new device 513 can be added to a device setup 511 by pressing the <Add 

20 New Device> button 547 on the Device Setup Details page 538. A Select Manufacturer 
page 548 is printed, as shown in Figure 27. The user selects the device manufacturer 549 
from the list by clicking the name of manufacturer. A Select Device Type page 550 is 
printed, as shown in Figure 28. 

The user selects the appropriate device type 551 from the list by clicking on the 
25 device type name. A Select Model page 552 is printed, as shown in Figure 29. 

The user selects the appropriate device model 553 from the list by clicking on 
the model name. The Set Device Defaults form 554 is printed, as shown in Figure 30. 

On the Set Device Defaults form 554 the user specifies an alias name for the 
device. The user then assigns this device as the default for one or more device types by 
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checking the appropriate check boxes. For example, the user's VCR may be assigned as 
the default television tuner and default video recorder in their device setup. When the 
form is complete the user presses <Add Device> 555. The Device Setup Details page 
538 is printed, showing the new device in the device list. 

5 The Add Device user interface flow is shown in Figure 1 9. 

User's Device List 

The user typically obtains the User's Device List 540 from the Device Setup 

main page 534 (<List My Devices> button 539). The user may also bookmark the page 
and subsequently obtain it via their favorites list, or obtain it via their history list. 

1 0 The Device List user interface flow is shown in Figure 1 8. 

The Device List 540 is shown in Figure 23 and indicates which devices are 
assigned as defaults for the setup. To print a Remote Control page 570 for a specific 
device the user clicks on the device name. An example of a device-specific television 
remote control 570 is shown in Figure 25. A device-specific remote control 570 will only 
15 operate the device named at the top of the remote control. A device-specific remote 
control for a particular television will only operate that one television, and will not 
operate any other television. 

The user can press the <Create URC> button 556 to print a Create Universal 
Remote Control form 557, as shown in Figure 24. This form lists all the devices in the 
20 cvirrent device setup. The user selects the devices to include on the URC by checking the 
appropriate device check boxes, and then clicking the <Print URC> button 558. The 
Universal Remote Control page 571 is printed. An example of a URC page 571 is shown 
in Figure 26. 

The user can press the <Device Setup> button 559 on the Device List Page 540 
25 to print their Device Setup Details page 538. 

The user can press the <Modify Defaults> button 560 to print a Modify 
Defaults form 561. This form allows the user to change the assignment of default 
devices. 

The user can press the <List Requests> button 562 to print a Request List 
30 page 563. This lists all the video broadcasts scheduled to be recorded for the user's 
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device setup. 

Rennote Controls 

There are three types of remote control. These are the device-specific remote 

control, the universal remote control^ and the generic remote control. Device specific and 
5 universal remote controls are printed by the user. Generic remote controls are printed in 
netpage publications such as television program schedules. 

The appearance and fimctions of a remote control v^ill depend on the device 
type, manufacturer etc. A manufacturer may provide their own netpage remote control 
w^hich can be printed by the user on demand. 

1 0 When the user presses a remote control button on a generic remote control the 

device control application looks up the appropriate default device in the user's device 
setup. This determines the user device and therefore the device model. 

A universal remote control 571 is illustrated in Figure 26. The device being 
controlled by a xmiversal remote control is determined by its mode (set v^hen a device 
1 5 alias name button 564 is pressed on the control). A device-specific remote control 570 as 
shown in Figure 25 is associated v^th a single user device. Thus when the user presses a 
remote control button 565 on a device-specific or universal remote control the device 
alias name, and therefore the device model, of the device being controlled is knovm. 

A remote control button 565 is associated with a remote control fimction 510, 
20 v^th a unique fimction identifier 517 (unique v^thin the device model). The device 
model is referenced in the manufacturer's database to look up the RCI (Remote Control 
Interface) fimction 516 control code 565 for the required fimction identifier. This 
command code is then transmitted to the user device. 

The method of the transmission may vary depending on the device's capability 
25 and the user's pen. 

Scheduled Broadcast Recording 

A netpage published television program schedule 572 contains a calendar of 

television broadcasts giving a description 566 of each program, and channel, date, and 
time information about the program. Each program listing may have associated with it 
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one or more buttons 567, as shown in Figure 31. 

The <Record> button 567 can be used to program the user's default video 
recording device to record the requested broadcast. The request 568 is stored in the 
user's program request Ust 573 (associated with their device setup). The Record Program 
5 user interface flow is shown in Figure 20. 

As a more advanced service, a program information provider can send updated 
program information to the user's printer, or can add program requests to the user's 
program request Ust 573 on a regular basis for a user's favorite programs. It can also 
automatically modify existing requests if program information changes. 

10 News Clips 

A published news story can have an associated video clip and printed playback 
control buttons. 

A set of news clips can be broadcast some time before the newspaper is 
delivered to the netpage printer 802. The printer 802 can be instructed by the pubUsher to 
1 5 Usten to the broadcast and record the broadcast on the user's disk-based video recording 
system. Clip indexing information is transmitted with the video clips, for example in the 
closed caption part of the broadcast. 

The buttons printed with the news story control playback of the broadcast video 
clip from the user's defauU disk-based video recorder to the user's default video monitor. 
20 Control buttons may include play, stop, slow motion, and freeze frame. 

Another type of video clip is the video-on-demand (VOD) video clip. A VOD 
video clip is broadcast to the user on demand, from a third party VOD supplier. 

Parental Control 

The Netpage device control system can also be used to give parental 

25 control over rated content. The system administrator can set the allowable content rating 
for particular users. A user without the required authority cannot operate a remote 
control to view rated chaimels or broadcasts. 

Conclusion 

The present invention has been described with reference to a preferred 
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embodiment and number of specific alternative embodiments. However, it will be 
appreciated by those skilled in the relevant fields that a number of other embodiments, 
differing from those specifically described, will also fall within the spirit and scope of 
the present invention. Accordingly, it will be understood that the invention is not 
intended to be limited to the specific embodiments described in the present specification, 
including documents incorporated by cross-reference as appropriate. The scope of the 
invention is only limited by the attached claims. 
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CLAIMS 

1 . A method of enabling a device to be controlled in which the device is operative 
to perform at least one function in response to control instructions from a computer 

5 system, the method including the steps of 

providing a control interface containing information relating to the device 
function, the control interface including coded data indicative of an identity of the 
control interface and of at least one reference point of the control interface; 

receiving, in the computer system, indicating data from a sensing device 
1 0 regarding the identity of the control interface and a position of the sensing device relative 
to the control interface, the sensing device, when placed in an operative position relative 
to the control interface, sensmg the indicating data using at least some of the coded data; 
and 

effecting, in the computer system and from the indicating data, an operation 
1 5 relating to at least one parameter of the control instructions. 

2. The method of claim 1 in which said at least one parameter relating to the 
control instructions is associated with at least one zone of the control interface and in 
which the method includes effecting, in the computer system and from the zone relative 

20 to which the sensing device is located, an operation relating to said at least one 
parameter. 

3 . The method of claim 2 which includes 

receiving, in the computer system, data regarding movement of the sensing 
25 device relative to the control interface, the sensing device sensing its movement relative 
to the control interface using at least some of the coded data; and 

effecting, in the computer system and from said movement being at least 
partially within said at least one zone, an operation relating to said at least one parameter 
of the control instructions. 
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4. A method of enabling a device to be controlled in which the device is operative 
to perform at least one function in response to control instructions from a computer 
system, the method including the steps of 

providing a control interface containing information relating to the device 
function, the control interface including coded data indicative of at least one parameter 
of the control instructions; 

receiving, in the computer system, data from a sensing device relating to said at 
least one parameter and regarding movement of the sensing device relative to the control 
interface, the sensing device, when moved relative to the control interface, sensing the 
data regarding said at least one parameter using at least some of the coded data and 
generating the data regarding its own movement relative to the control interface; and 

interpreting, in the computer system, said movement of the sensing device as it 
relates to said at least one parameter. 

5. A method of enabling a device to be controlled in which the device is operative 

to perform at least one function in response to control instructions from a computer 
system, the method including the steps of 

providing a user with a control interface including coded data indicative of an 
identity of the control interface; 

receiving, in the computer system, data from a sensing device regarding an 
identity of the user and regarding the identity of the control interface, the sensing device 
containing the data regarding the identity of the user and sensing the data regarding the 
identity of the control interface using at least some of the coded data; and 

effecting, in the computer system and from the data regarding the identity of the 
user and the identity of the control instructions, an operation relating to at least one 
parameter of the control instructions. 

6. The method of claim 6 which includes receiving, in the computer system, data 
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from the sensing device regarding movement of the sensing device relative to the control 
interface, the sensing device generating data regarding its own movement relative to the 
control interface. 

5 7, The method of any one of claims 1 , 4 or 5 in which the parameter of the control 
instructions is selected from the group comprising: 

selecting said device; 

selecting said fimction to be performed; 

establishing default setting for said function; 

1 0 * establishing default settmg for said device; 

registering user access to control the device function; 

authorising user access to control the device function; and 

issuing a command code to said device to perform said function. 

15 8. The method according to any one of claims 1, 4 or 5 further including the step 
of issuing a command code to said device to perform said function in response to 
operation of the computer system. 

9. The method of claim 8 in which the command code is issued to said device 
20 through said sensing device. 

10. The method of claim 8 in which the command code is issued to said device 
independently of said sensing device. 

25 11. The method according to claim 8 in which the command code is issued to said 
device using wireless technology. 
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12. The method of any one of claims 1, 4 or 5 which includes printing the control 
interface on demand. 

13. The method of claim 12 which includes printing the control interface on a 
5 surface of a surface-defining means and, at the same time that the control interface is 

printed, printing the coded data on the surface. 

14. The method of claim 13 which includes printing the coded data to be 
substantially invisible in the visible spectrum. 

10 

15. The method of claim 1, 4, or 5 which includes retaining a retrievable record of 
each control interface generated, the control interface being retrievable using its identity 
as contained in its coded data. 

15 16. The method of any one of claims 1, 4 or 5 which includes distributing a 
plurality of the control interfaces using a mixture of multicast and pointcast 
communications protocols. 

17. The method of claim 1 or 4 in which the sensing device contains an 
20 identification means which imparts a unique identity to the sensing device and identifies 

it as belonging to a particular user and in which the method includes monitoring, in the 
computer system, said identity. 

18. The method of any one of claims 1, 4 or 5 which includes providing all required 
25 information relating to the device function in the control interface to eUminate the need 

for a separate display device. 

19. The method of any one of claims 1, 4 or 5 in which the control interface is 



NPA059US 



-38- 

printed on multiple pages and in which the method includes binding the pages. 



20. A system for enabling at least one function of a device to be controlled, the 
system including 

5 a control interface containing information relating to the device function, the 

control interface including coded data indicative of an identity of the control interface 
and of at least one reference point of the control interface; and 

a computer system incorporating control instructions operative to cause said 
device to perform said function and operative to receive indicating data from a sensing 
10 device for effecting an operation relating to at least one parameter of the control 
instructions, the indicating data being indicative of the identity of the control interface 
and a position of the sensing device relative to the control interface, the sensing device 
sensing the indicating data using at least some of the coded data. 

15 21. The system of claim 29 in which said at least one parameter relating to the 
control instructions is associated with at least one zone of the control interface. 

22. The system of claim 29 which includes the sensing device, the sensing device 
sensing its movement relative to the control interface using at least some of the coded 

20 data. 

23. A system for enabling at least one function of a device to be controlled in 
response to control instructions, the system including 

a control interface containing information relating to the device function, the 
25 control interface including coded data indicative of at least one parameter of the control 
instructions; and 

a computer system incorporating said control instructions, and operative to 
receive data from a sensing device regarding said at least one parameter and regarding 
movement of the sensing device relative to the control interface, and for interpreting said 
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movement of the sensing device as it relates to said at least one parameter, the sensing 
device, when moved relative to the control interface, sensing the data regarding said at 
least one parameter using at least some of the coded data and generating the data 
regarding its own movement relative to the control interface. 

5 

24. A system for enabling at least one function of a device to be controlled, the 
system including 

a control interface including coded data indicative of an identity of the control 
interface; and 

10 a computer system incorporating control instructions to cause said device to 

perform said function, and operative to receive from a sensing device data regarding an 
identity of the user and the identity of the control interface, and for effecting, from said 
received data, an operation relating to a parameter of said control instructions, the 
sensing device containmg the data regarding the identity of the user and sensing the data 

1 5 regarding the identity of the control interface using at least some of the coded data. 

25. The system of claim 24 which includes the sensing device, the sensing device 
sensing its movement relative to the control interface. 

20 26. The system of any one of claims 20, 23 or 24 in which the parameter of the 
control instructions is selected from the group comprising 

selecting said device; 

selecting said function to be performed; 

estabhshing default setting for said function; 

25 • establishing default setting for said device; 

registering user access to control the device function; 

authorising user access to control the device function; and 

issuing a command code to said device to perform said fimction. 
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27. A system according to any one of claims 20, 23, or 24 in which the computer is 
operative to issue a command code to said device to perform said function through said 
sensing device. 

5 

28. A system according to any one of claims 20, 23, or 24 in which the computer is 
operative to issue a command code to said device to perform said function independently 
of said sensing device. 

10 29. The system of claim 20 or 23 in which the sensing device contains an 
identification means which imparts a unique identity to the sensing device and identifies 
it as belonging to a particular user. 

30. The system of any one of claims 20, 23, or 24 in which the control interface is 
1 5 printed on a surface of a surface-defining means and in which the system includes a 

printer for printing the control interface on demand. 

31. The system of claim 30 in which the printer prints the coded data at the same 
time as printing the control interface on the surface-defining means. 

20 

32. The system of any one of claims 20, 23, or 24 in which the coded data is 
substantially invisible in the visible spectrum. 

33. The system of claim 20 or 23 which includes a database for keeping a 
25 retrievable record of each control interface generated, each control interface being 

retrievable by using its identity as included in its coded data. 

34. The system of claim 30 in which, to cater for a control interface printed on 
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multiple pages, the printer includes a binding means for binding the pages. 
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ABSTRACT 



The invention concerns device control for enabling electrical or electronic 
devices to be controlled through computer systems. Employing the invention involves 
5 the use of one or more control interfaces which are capable of interacting with a 
computer system or network, each interface comprising sheet material such as paper 
which has coded data printed on it and which allows it to interact with the computer 
system by use of a sensing device operated by a user. Through this interaction, data can 
be forwarded to the computer system from the control interface to effect in that system 
10 an operation relating to the functioning of the device. 



(Figure 8) 



1/29 




■ 



NAME 
TEXT 



ADDR 
TEXT 

SUBMIT 
GRAPHIC 



7 




TITLE 
TEXT 



NAME 
TEXT FIELD 

ADPR1 
TEXT FIELD 

ADDR 2 
TEXT FIELD 

SUBMIT 
BUTTON 



6 



FIG. 1 



2/29 



NETPAGE 
APPLICATION 
SERVER 



13 



NETPAC5E 
PAGE SERVER 



10 



NETPAGE 
PRINTER 




9 



FIG. 2 



3/29 



NETFAGE PAGE 
-10- 



CONTENT 
OBJECT 



DOCUMENT 
DESCRIPTION 



o 



PAGE 
DESCRIPTION 



GRAPHIC 
ELEMENT 



INTERACTIVE 
ELEMENT 



DOCUMENT 
INSTANCE 



NETPAGE 
DOCUMENT 



o 



PAGE 
INSTANCE 



PAGE ID <- 




NETPAGE 



FIELD 
VALUE 



50 



2- 



o 



VISIBLE 
GRAPHIC DATA 



INVISIBLE 
CODED DATA 



50 



PHYSICAL WORLD 



4 



INVISIBLE 
TAG 



PAGE ID ^ 
POSITION 



FIO. 3 



4/29 

4-^ 




FIG. 4\? 



5/29 




FIG. 5b 



6/29 



•••••••••••••••••••••••••••••••••••••••••••••••• 



•••••••••••••••••••••••••••••••••••••••••••••••• 



47 





1A 


O 


1B 




1C 


o 




OS 


I— 




IV) 
IL 




ro 


12 


ro 
> 


o 


1D 




1E 




IF 










re 


ro 




ro 
m 


H2 






1H 




11 




1J 




1K 




ro 
z: 




ro 
c_ 




ro 
-n 


a2 




< 


IL 




1M 




1N 


1 


10 




DO 
O 


02 


ro 

7^ 




ro 


V2 





FIO. 5d 



7/29 



GENERIC 
DOCUMENT 



GENERIC 
FORM 



USER 
DOCUMENT 



USER 
FORM 



990 



991 



992 



993 



FIG. 6 



FIXED INFORMATION 994 

\ VARIABLE INFORMATION 995 



C COMMAND BUTTON 996 

^997 



INPUT FIELD 



DRAGGABLE COMMAND 



993 



FIO. 7 



8/29 



503 




500 



FIG. & 



9/29 



521 



511 



PEVICE 




SETUP 




SETUP NAME-^ 



512 



524- 



PROGRAM 
REQUEST 



REQUEST IP 
DESCRIPTION 



DEFAULT I 
DEVICE 1 



DEVICE 
MODEL 



56& 



USER 

"S" 

&00 



! *■ 



' PROGRAM ' 
; SCHEDULE \ 
I ENTRY I 



529 



513 





USER 




DEVICE 


ALIAS NAME>^ 



NETPAGE 
PRINTER 



I NETPAGE 1 



^302 
515 



514 



MODE 



REMOTE 
CONTROL 



510' 



REMOTE 
CONTROL 
FUNCTION 



FUNCTION ID 



517 



FIG. 9 



10/29 




52/ 



DEFAULT /\ 
PEVICE ST 



VIDEO 
RECORDER 



VIDEO 
MONITOR 



AUDIO 
RECEIVER 



AUDIO 
RECORDER 



AUDIO 
MONITOR 



FIG. 10 



11/29 



MANUFAC- 
TURER 



3e& 



PEVICE 



569 



DEVICE 
MODEL 



CDTL 

CAPABILITY- 



REMOTE 
CONTROL 
INTERFACE 



RCl 
FUNCTION 



FUNCTION ID 
CONTROL CODE 



524 



526 



■516 



565 



527 



. CHANNEL 
"' LOCALITY 





DVD PLAYER 








TELEVISION 








VCR 








DI5K-3A5ED 

VIDEO 
RECORDER 








RADIO 
RECEIVER 








CD PLAYER 








AMPLIFIER 





53; 



FIG. 11 



12/29 



DEVICE 



569 



t 


CHANNEL 


1 ^ 


LOCALITY 









CHANNEL 
CATEGORY 



53f 



552 




REMOTE 
CONTROL 



515 





PEVICE- 
5FEC1FIC RC 








UNIVERSAL RC 








GENERIC RC 





518> 



519 



520 



FIO. 15 



13/29 



TELEVISION 
PROGRAM 
SCHEDULE 



PROGRAM 
SCHEDULE 
ENTRY 



LISTING TEXT 



52& 



329 



CDTL 
INFORMATION 



330 



FIG. 14 



14/29 



534 



PEVICE 
SETUP 
MAIN PAGE 



DETAlie 



NEW PEVICE 
SETUP 



533 



PEVICE 
SETUP 
DETAILS 



540 



LI5TMY 
DEVICE3 



USER'S 
PEVICE 
LIST 



FIO. 15 



15/29 



DEVICE 
SETUP 
DETAILS 



G5 



53& 

U5EK 
DETAILS 



REMOVE DEVICE 
/ REMOVE USER 



542 



USER 
DETAILS 



DEVICE 
DETAILS ^ 



543 



DEVICE 
DETAILS 



ADD NEW 
USER 



546 



ADD 
USER 
TO SETUP 



ADD NEW 
DEVICE 



543 



SELECT 
MAN U FACT 
-URER 



FIG. 16 



16/29 



540 



USER'S 
DEVICE 
LIST 



DEVICE 



570 



DEVICE 
REMOTE 
CONTROL 



57/ 



CREATE 
UKC 



KE5EV 
FORM 



CREATE 
URC 



(5 



557 

PRINT 
URC . 



UNIVERSAL 
REMOTE 
CONTROL 



MODIFY 
DEFAULTS 
► 


MODIFY 
DEFAULT 
DEVICES 


r 





561 



RE5E1 
FORM 



LIST 
REQUESTS 
H 



MODIFY 



563 



PROGRAM 
REQUEST 
LIST 



DEVICE 
SETUP 



■533 



DEVICE 
SETUP 
DETAILS 



FIG. 17 



17/29 



535 



PEVICE 
SETUP 
DETAILS 



54& 



SELECT 
MANUFACT 
-URER 



MANU - 
FACTURED 



550 



SELECT 
DEVICE 
TYPE 



DEVICE TYPE 
■552 



SELECT 
MODEL 



A9P PEVICE 



MODEL 



554 



SET 
DEVICE 
DEFAULTS 



FIG. 13 



18/29 



572 



TELEVISION 
PROGRAM 
SCHEDULE 



540 



USER'S 
DEVICE 
LIST 



LIST 
F:BQUE5T5 



RECORD 



575 



PROGRAM 
REQUEST 
LIST 



CANCEL 



FIG. 19 



19/29 



PEVICE SETUF MAIN FAGB ^^^^ ^ ^ 

CUKKEHT DEVICE SETUF5 FO^ THIS FRINTER: 



( PETAILS ) I SETUP NAME \ 1 USER NAME ] 

; USER NAME \ 

k — _ — — ^ — ~ — — — — — J 

( PETAILS ) ! SETUP NAME ! [ USER NAME ] 

; USER NAME ! 

a/CK ^?Ay PETAHa (A30VE) to view of modify device 5ETUF 
OF 

ENTER 5ETUF NAME (3EL0W) TO CFEATE NEW DEVICE 5ETUF 



SETUP NAME 



( NEW DEVICE SETUP ) 
536 5-37 



534 

539 



LIST MY devices"^ 



FIG. 20 



20/29 



PEVICE SETUF PETAIL5 


r/\c5E / OF 1 


\ SETUP NAME ! 




*— 1 -^f— *o ^— / J ./^ 

REGISTEREO U3Ek5: 




! USER NAME 


\ ( DETAILS } C REMOVE ) 

. — mm J 


; USER NAME 


\ ( DETAILS ) ( REMOVE ) 

. _ _ J ^^^^^^^^^^^ 




544 


; USER NAME 


! C PETAILS 3 C REMOVE ) 




^ r ^ 

54? ( ADD NEW USER J 


PEVICEB: 


543 


; PEvlCE ALlAb 


' r DETAILS r REMOVE ^ 


; DEVICE ALIAS 


\ C DETAILS 3 C REMOVE ) 




544 


; DEVICE ALIAS 


\ C DETAILS } C REMOVE ) 




Cadd new device) 




547 



FIG. 21 



21/29 



DEVICE U5r FOR \ 


^" "user name" ; PAOEWF1 


FKE55 A DEVICE 3UTT0N TO FKINT A REMOTE CONTROL FAGE 


\ DEVICE ALIAS 


\ PEFA UL T YIPEO KEC ORDER 


: DEVICE ALIAS 


\ DEFAULT VIDEO MONITOR 


: DEVICE ALIAS 


! TELEVISION 


I 

\ DEVICE ALIAS 


— -I 
1 

_ J 


C CREATE URC 


) CREATE UNIVERSAL REMOTE CONTROL 


556 






562 L15-P REQUESTS ) 




MODIFY DEFAULTS ) 




559 -^^-(^ DEVICE SETUP ) 



FIG. 22 



22/29 



CREA TE UNIVER5AL REMOTE CONTROL ^^^^ ^ ^ 

r — — — — — — — — — — — — , 

CURRENT DEVICE 5ETUF: I SETUP NAME 



. — — — — — J 



CHECK DEVICES REQUIRED ON UNIVERSAL REMOTE CONTROL 



DEVICE ALIA5 


! ; DEVICE TYPE 




PEVICE ALIAS 


; ! DEVICE TYPE 




DEVICE ALIAS 


! ; DEVICE TYPE 



553 



PRINT URC 



( RESET fOm) 



FIG. 23 



23/29 



; DEVICE ALIAS 



FAGE10F1\ PATE & TIME; 



L — — — — — — • 



REMOTE CONTROL 



( TV / VIDEO ) 



( POWER ON/OFF ) 



565 



(JD CU Caj 



CZD CO CID 



CZ3 CZD CO 



GIZ3 



( MUTE ) 



VOLUME 



CHANNEL 



CZJ 



C REPRINT ) ( PRINT TV PROGRAM ) ( LIST PEVICES ) 



24/29 



UNIVERSAL REMOTE ^^G^ tofi\ We & time ; 
CONTROL ' 



DEVICE ALIAS 


! : DEVICE ALIAS 


DEVICE ALIAS 


: : DEVICE ALIAS 


DEVICE ALIAS 


: : DEVICE ALIAS 



564 



( POWER ON/OFF ) 





(-) ( 


ai 


) C ■ ) 




505 
















® 




C ^ ) 


( 1 ) 


C 2 ) 


( 3 ) 


( - ) 


CHANNEL 


( 4 ) 


( 5 ) 


( 6 ) 


VOLUME 


C - ) 








( - ) 




( 7 ) 


C » ) 


C 9 ) 








C 0 ) 







( MUTE ) 



C REPRINT ) ( PRINT TV PROGRAM ) ( LIST DEVICES ) 



FIO. 25 



25/29 



SELECT MANUFACTURER \Z^p}}~9fjt\ 

r 

APDINO NEW DEVICE TO 5ETUF: [ _5ETUP N_AME 1 

SELECT MANUFACTURER OF PEVICE 
■ MANUFACTURER \ 

u . 

[ manufacturer ; 

I. •» 

; MANUFACTURER 54^ 

b 

___J 

: MANUFACTURER \ 



543^ 



FIG. 20 



26/29 



5ELECr PBVICE TYFE ^^^^ ^ OF 1 \ tkah H3K ; 



ADDING NEW DEVICE TO 5ETUF: I SETUP NAME 

— — — — — — — 

SELECT \ M AN UFAcfuRER' " " DEVICE TYPE 



( TELEVISION ) 
( VCR ) 

C PVQ PLAYER ) 
( AMPLIFIER ) 
C CP PLAYER ) 
(PISK-gASEP VIDEO REC) 
(RAPIO receiver) 



551 



550' 



FIG. 27 



27/29 



SBLECrMOPEL OF DEVICE [ ^ageI^of ] m^^^^ 

r ■» 

APPING NEW DEVICE TO 5ETUF: [ _^_l'[^f_ |^A_ME 1 

___ _ _ _ — *n p.— — •» — — — — — — — — T 

SELECT MODEL OF \ MANUFACTURER ! DEVICE TY_P_E_ ! 



MODEL 



MODEL 



MODEL 



MODEL 



FIG. 23 



28/29 



5Er PBVICB PBFAULTS [ !^'^(5E i OF _ T5_AN NB^^^ 

r -1 

ADDING NEW DEVICE TO 5ETUF: 1 _5ETUP NAME 1 

ENTEK DEVICE ALIA5 FOK 

— — — — — — — — — — — — — 1 

\uhm?hoi\ji£i~ DEVICE TYPE :; model 



DEVICE ALIAS 



DO YOU WANT THIS TO 3B YOUR DEFAULT 

YES 

; "default DEVICE TTFE ? □ 

1. -« 

i'dEf'auYt" DEVICE TYPE 1 ? □ 

L 

! 'dEf'aULT device TYPE 1 ? □ 

I. : ■» 

_A...... 

; DEFAULT DEVICE TYPE 7 □ 

L ■» 

(CHECK All THATAFFIY) 



555-^^ ^ ADD DEVICE ) 



FIO. 29 



29/29 



TELEVISION FKOGKAM 5CHEPULE 
FOK ! PATE : 



; DATE & time; 
; 'pageTof #; 




(REC) 



PROGRAM LISTING 
ENTRY 



567 



566 



572 



FIG, 50 



Please type a plus sign {+) inside this box — > 



PTO/SB/01 (12-97) 
Approved for use through 9/30/00. OMB 0651-0032 
Patent and Trademark Office, U.S. DEPARTMENT OF COMMERCE 
Under the Paperworl< Reduction Act of 1995, no persons are required to respond to a collection of information unless it contains 
a valid OMB control number. 



-I- 



DECLARATION FOR UTILITY OR 
DESIGN 
PATENT APPLICATION 

(37 CFR1.63) 



H Declaration 
Submitted 
with Initial 
Filing 



□ Declaration 
OR Submitted after Initial 
Filing (surcharge 
(37 CFR 1.16(e)) 
required) 



Attorney Docket Number 



First Named Inventor 



NPA059US 



PAULLAPSTUN 



COMPLETE /F KNOWN 



Application Number 



Filing Date 



Group Art Unit 



Examiner Name 



/ 



As a below named inventor, I hereby declare that: 

My residence, post office address, and citizenship are as stated below next to my name. 

i believe I am the original, first and sole inventor (if only one name is listed below) or an original, first and joint inventor (if plural 
names are listed below) of the subject matter which is claimed and for which a patent is sought on the invention entitled. 



METHOD AND SYSTEM FOR DEVICE CONTROL 



the specification of which 

^ IS attached hereto 
OR 

□ was filed on (MM/DD/YYYY) 



(Title of the Invent/on) 



n and was amended on (MM/DD/YYYY) 



as United States Application Number or PCT Internationa! 

I (if applicable). 



Application Number | 

I hereby state that 1 have reviewed and understand the contents of the above identified specification, including the claims, as 
amended by any amendment specifically referred to above. 

1 acknowledge the duty to disclose information which is matenal to patentability as defined in 37 CFR 1.56. 



hereby claim foreign priority benefits under 35 U.S.C, 119(a)-(d) or 365(b) of any foreign applicatton(s) for patent or inventors 
certificate or 365(a) of any PCT international application which designated at least one country other than the United States of 
Amenca listed below and have also identified below, by checking the box, any foreign application for patent or inventors certificate, 
or of any PCT international application having a filing date before that of the application on which priority is claimed. 



Prior Foreign Application 
Nunriber(s) 



Country 



Foreign Filing Date 
(IVilVI/DD/YYYY) 



Priority 
Not Claimed 



Certified Copy Attached? 
YES NO 



PQ4392 



Australia 



12-01-1999 



□ 
□ 
□ 
□ 



□ 
□ 
□ 



□ 
□ 
□ 
□ 



□ Additional foreign application numbers are listed on a supplemental priority data sheet PTO/SB/02 B attached hereto: 

i i . ._• _ I £.1 I ie 1 1 o ^'1Q/«\ ono I XnWaA Qfatoc nrnvicinnal annliratinn^R^ listed below. 



hereby claim the benefit under 35 U.S.C. 119fe^ of any United States provisiona l application(s) listed below. 



Application Number(s) 



Filing Date (MIVI/DP/YYYY) 



I I Additional provisional application 
numbers are listed on a 
supplemental priority data sheet 
PTO/SB/02B attached hereto. 



+ 



[Page 1 of 2] 

Burden Hour Statement. This form is estimated to take 0.4 hours to complete. Time wiii vary depending upon tj^e needs of the 
individual case. Any comments on the amount of time you are required to complete this form should be sent to the Chief Information 
Officer. Patent and Trademark Office. Washington, DC 20231. DO NOT SEND FEES OR COMPLETED FORMS TO THIS 
ADDRESS SEND TO. Assistant Commissioner for Patents. Washington, DC 20231 . 



DocketNo.: NPA059US 



HPTO/SB/01 (12-97) I 
Approved for use through 9/30/00. OMB 0651-0032 I 
Patent and Trademark Office; U.S. DEPARTMENT OF COMMERCE 
Under the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless it contains 
a valid OMB control number. „^ 



Please type a plus sign {+) inside this box 



DECLARATION — Utility or Design Patent Application 



1 hereby claim the benefit under 35 U.S.C. 120 of any United States appiication(s), or 365(c of any PCT international aPPl/c^tion designating th^^ 
United States of Amenca, listed below and, insofar as the subject matter of each of the claims of this application is not disclosed in the pnor 
United States or PCT International application in the manner provided by the first paragraph of 35 U.S.C. 1 12 I fc^^noy^ ^dge me dut^ 1° ^ S^hnn 
information which is material to patentability as defined in 37 CFR 1.56 which became available between the filing date of the pnor application 
and the national or PCT international filing date of this application. 



U.S. Parent Application or PCT Parent 
Number 



Parent Filing Date 
(MM/DD/YYYY) 



Parent Patent Number 
(if applicable) 



] Additional U.S. or PCT international application numbers are listed on a supplemental priority data sheet PTO/SB/02B attached hereto. 

business in the Patent 



As a named inventor, I hereby appoint the following registered practi tionerfs) to prosecute this app lication and to trai isact 
and Trademark Office connected therewith. Q Customer Number [ 

OR 



□ Registered practitioner(s) name/registration number listed below 



Place Customer 
Number Bar Code 
I ahftf hfira 



Name 



Registration 
Number 



Name 



Registration 
Number 



Additional registered practitioner fs) named on supplemental Registered Practitioner Information sheet PTO/SB/02C attached hereto. 



Direct all correspondence to: H Customer Number 

or Bar Code Label 



24011 



OR S Correspondence address below 



Name 


Kia Silverbrook 


Address 


Silverbrook Research Pty Ltd 


Address 


393 Darling Street 


City 


Balmain 


State 


NSW 


ZIP 


2041 


Country 


Australia | Telephone 


61'2~9818-6633 


Fax 


61-2-9818-6711 



I herebv declare that ail statements made herein of my own knowledge are true ana mat an bid «(i em& nicjuc u.i hi.wmmcuw , ^..w. 
believed to be true and further that these statements were made with the knowledge that willful false statements and the like so rnade are 
punlhable by fine of impnsonment, or both, under 18 U.S C. 1001 and that such willful false statements may jeopardize the validity of the 
application or any patent issued thereork - - 



Nanne of Sole or First Inventor: 



□ A petition has been filed for this unsigned inventor 



Given Name (first and middle fif anvl) 



Family Nap n^ nr g^iirnannR 



PAUL 



n 



LAPSTUN 



Oct 18, 
2000 



Inventor's 
Signature 



Date 



Residence: City 



Rodd Point 



state 



NSW 



Country 



Australia 



Citizenship 



Norwegii 



Post Office Address 



13 Duke Avenue 



Post Office Address 



City 



Rodd Point state! 



ZIP 



2046 



I 



Country 



Australia 



El Additional inventors are being named on the _-?.__supplemental Additional Inventor(s) sheet(s) PTOySB/02A attached heretd 



+ 



[Page 2 of 2] 



Docket No. NPA059US 



Please type a plus sign (+) inside this box 



PTO/SB/02A (3-97) 
Approved for use through 9/30/98. OMB 0651-0032 JL^ 
Patent and Trademark Office, U.S. DEPARTMENT OF COMMERCE | 
Under the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless It contains a 
valid OMB control number. ^ 



DECLARATION 



ADDITIONAL INVENTOR(S) 
Supplemental Sheet 

- ijL_of_Z„ 



Name of Additional Joint Inventor, if any: 



ri A petition has been filed for this unsigned inventor 



Given Name (first and nniddle [If any]) 



Family Name or Surname 



JACQUELINE ANNE 



LAPSTUN 



Inventor's 
Signature 




Date 



Oct. 18, 
2000 



Residence: City 



Rodd Point 



state 



NSW 



Country 



Australia 



Citizenship 



Australian 



23 Duke Avenue 



Post Office Address 



Post Office Address 



City 



Rodd Point 



state 



Name of Additional Joint inventor, if any: 



NSW 



ZIP 



2046 



Country Australia 



n A petition has been filed for this unsigned inventor 



Given Name (first and middle [If any]) 



Family Name or Surname 



KIA 



SILVERBROOK 



Inventor's 
Signature 



Date 



Oct. 18, 

2m 



Residence: City 



Balmain 



state 



NSW 



Country 



Australia 



Citizenship 



Post Office Address 



393 Darling Street 



Post Office Address 



City 



Balmain 



state 



NSW 



ZIP 



2041 



Country 



Australia 



Name of Additional Joint Inventor, if any: 



n A petition has been filed for this unsigned inventor 



Given Name (first and middle [if any]) 



Family Name or Surname 



Inventor's 
Signature 



Date 



Residence: City 



state 



Country 



Citizenship 



Post Office Address 



Post Office Address 



City 



state 



Country 



+ 



Burden Hour Statement: This form Is estimated to take 0.4 hours to complete. Time will vary depending upon the needs t>ie mdual case^^^^ 
mmments on the amount of time you are required to complete this form should be sent to the Chief Information Officer, Patent and Traden^ark 
Se Washington DC 2023^^ SEND FEES OR COMPLETED FORMS TO THIS ADDRESS. SEND TO. Assistant Commiss.oner for 

Patents, Washington, DC 20231. 



